Corpus-based syntax-prosody tree matching

Empirical study of the syntax-prosody relation is hampered by the fact that current prosodic models are essentially linear, while syntactic structure is hierarchical. The present contribution describes a syntax-prosody comparison heuristic based on two new algorithms: Time Tree Induction, TTI, for building a prosodic treebank from time-annotated speech data, and Tree Similarity Indexing, TSI) for comparing syntactic trees with the prosodic trees. Two parametrisations of the TTI algorithm, for different tree branching conditions, are applied to sentences taken from a read-aloud narrative, and compared with parses of the same sentences, using the TSI. In addition, null-hypotheses in the form of flat bracketing of the sentences are compared. A preference for iambic (heavy rightmost branch) grouping is found. The resulting quantitative evidence for syntax-prosody relations has applications in speech genre characterisation and in duration models for speech synthesis. 1. Hierarchical syntax, linear prosody? The objective of this contribution is to provide a well-defined algorithmic approach to extracting complex prosodic information from speech corpora. Current empirical models of speech timing are based on a variety of algorithms, from single indices for timing patterns [1, 2] in psycholinguistics and phonetics to, in the speech synthesis domain, sum-of-products, CART and Bayesian classification approaches [3, 4], including models which use grammatical information. Campbell [5] has a model based on a strictly layered hierarchy, but in general duration models are linear, and hold for flat strings of words or syntactic categories. When syntagmatic grammatical information is used as a predictor for hierarchical structuring, in general the information used is also linear, based on strings of paradigmatic part-of-speech (POS) classes (grammatical categories) which provide weight factors for duration models. Wagner [6] uses a linear model for German speech synthesis based on five weighted POS sets. Grammatical categories imply at least local “hidden hierarchies”, of course. Rarely, explicit hierarchical approaches have been used [7], and detailed approaches to the partially hierarchical description of timing are once more becoming available [8], [9, 10, 11]; But there is currently no technique available for data-driven investigation of more complex hierarchical duration models for syntagmatic prosodic relations, and the issue is not addressed in recent authoritative literature [12]. Classification methods need 1Thanks to Grazyna Demenko, Katarzyna Dziubalska-Kołaczyk, Ekaterina Iassinskaia, Peter Ladkin, Zofia Malisz and lecture audiences in Dublin, Bielefeld, Poznań and Tübingen for discussion and to Ulrike Gut, Katrin Johanning, Sara Johannsen, Josef Raab, Alexandra Thies, Thorsten Trippel for contributing data. The software developed for this work is in the public domain (GPL). to include complex hierarchical timing information in addition to other to other phonetic and lexical properties of speech units. Further, it is a well-known phonostylistic effect that speech timing relations vary in highly complex ways depending on speech genre, including so-called fast speech phenomena [13]. Finally, other discourse factors such as focus and emotion are thought to affecting prosody, and thereby reducing the determining role of phrasal syntax, though these effects are currently not well understood (but see [14]). 2. Linear timing measures One set of approaches to investigating syntagmatic properties of timing is found in phonetic analyses of isochrony in syllable and foot timing. In [15], tone unit duration is divided by the number of feet in the tone unit, yielding average or “ideal” isochronous foot duration, and normalised deviation from mean foot length is measured. Neither hierarchy nor linear alternation of timing units figure in the approach, which may be said to use a Global Evenness (GE) criterion as a measure of the isochrony property, rather than the alternation property. Any arbitrary re-sorting of the relevant segments in an utterance (random, shortest-tolongest, etc.) would yield the same index. Timing fulfils the GE criterion, in some sense, but it has other properties too, so while the GE criterion for timing is a necessary criterion for isochrony it is (going beyond Roach’s stated goals, of course) not a sufficient criterion for an adequate timing model. Ramus, Nespor & Mehler [2] locate different languages in a timing space with the following parameters: , percentage of vocalic intervals relative to overall utterance length; , variance of consonantal intervals; , variance of vocalic intervals. The model also uses a variety of GE criterion: V stretches and C stretches would still yield the same results if randomly sorted (by length, longer consonant sequences first, etc.). Similar considerations apply to the measure, which reflects evenness of vowel sequence lengths, lower values tending to isochrony, and to the measure. The model does not have hierarchical and alternating timing components and is thus is incomplete as a model of rhythm timing, though it is claimed to be a model of rhythm. Cummins has pointed out [9] that the model makes a statement about the evenness of the phonotactics of the language, rather than timing. The model possibly reflects necessary conditions on timing, but falls short of providing a sufficient condition. Low, Grabe & Nolan [1] addressed the GE issue and developed the Pairwise Variability Index (PVI) in order to take iterative alternation into account. The PVI measures normalised differences between the durations of adjacent units (vowels, syllables, etc.):